AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.495855	1.81
	2_DecisionTree	Decision Tree	logloss	0.220615	18.58
	3_Default_Xgboost	Xgboost	logloss	0.195571	19.53
	4_Default_NeuralNetwork	Neural Network	logloss	0.217379	11.23
	5_Default_RandomForest	Random Forest	logloss	0.210298	22
the best	Ensemble	Ensemble	logloss	0.19518	1.59

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

9.8 seconds

Metric details

	score	threshold
logloss	0.217379	nan
auc	0.952569	nan
f1	0.941307	0.390958
accuracy	0.904342	0.390958
precision	0.99935	0.993516
recall	1	3.78385e-07
mcc	0.701957	0.63984

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.217379	nan
auc	0.952569	nan
f1	0.941307	0.390958
accuracy	0.904342	0.390958
precision	0.928044	0.390958
recall	0.954955	0.390958
mcc	0.685234	0.390958

Confusion matrix (at threshold=0.390958)

	Predicted as Major	Predicted as Minor
Labeled as Major	607	263
Labeled as Minor	160	3392

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

17.0 seconds

Metric details

	score	threshold
logloss	0.220615	nan
auc	0.943118	nan
f1	0.891007	0
accuracy	0.803483	0
precision	0.992863	0.985129
recall	1	0
mcc	0.598731	0.831642

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.220615	nan
auc	0.943118	nan
f1	0.891007	0
accuracy	0.803483	0
precision	0.803438	0
recall	1	0
mcc	0.030389	0

Confusion matrix (at threshold=0.0)

	Predicted as Major	Predicted as Minor
Labeled as Major	1	869
Labeled as Minor	0	3552

Learning curves

Decision Tree

Tree #1

Rules

if (APR Severity of Illness Code <= 2.5) and (Age Group <= 3.5) and (APR Severity of Illness Code > 0.5) then class: Minor (proba: 99.38%) | based on 7,565 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description > 0.5) and (Age Group <= 3.5) then class: Minor (proba: 66.95%) | based on 1,649 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group > 3.5) and (APR Severity of Illness Code > 1.5) then class: Minor (proba: 82.68%) | based on 1,380 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description > 0.5) and (Age Group > 3.5) then class: Major (proba: 74.2%) | based on 1,341 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description <= 0.5) and (APR MDC Description <= 17.5) then class: Major (proba: 96.08%) | based on 739 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group > 3.5) and (APR Severity of Illness Code <= 1.5) then class: Minor (proba: 97.65%) | based on 510 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description <= 0.5) and (APR MDC Description > 17.5) then class: Major (proba: 77.5%) | based on 80 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group <= 3.5) and (APR Severity of Illness Code <= 0.5) then class: Major (proba: 100.0%) | based on 1 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: binary:logistic
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

17.8 seconds

Metric details

	score	threshold
logloss	0.195571	nan
auc	0.961201	nan
f1	0.947646	0.463998
accuracy	0.914744	0.463998
precision	1	0.996263
recall	1	0.00113379
mcc	0.725176	0.643173

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.195571	nan
auc	0.961201	nan
f1	0.947646	0.463998
accuracy	0.914744	0.463998
precision	0.935051	0.463998
recall	0.960586	0.463998
mcc	0.720305	0.463998

Confusion matrix (at threshold=0.463998)

	Predicted as Major	Predicted as Minor
Labeled as Major	633	237
Labeled as Minor	140	3412

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

0.8 seconds

Metric details

	score	threshold
logloss	0.495855	nan
auc	0.5	nan
f1	0.890895	0.72285
accuracy	0.803256	0.72285
precision	0.803256	0.72285
recall	1	0.72285
mcc	0	0.72285

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.495855	nan
auc	0.5	nan
f1	0.890895	0.72285
accuracy	0.803256	0.72285
precision	0.803256	0.72285
recall	1	0.72285
mcc	0	0.72285

Confusion matrix (at threshold=0.72285)

	Predicted as Major	Predicted as Minor
Labeled as Major	0	870
Labeled as Minor	0	3552

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	4
4_Default_NeuralNetwork	1

Metric details

	score	threshold
logloss	0.19518	nan
auc	0.961384	nan
f1	0.947502	0.530245
accuracy	0.915649	0.530245
precision	1	0.997066
recall	1	0.0045363
mcc	0.733011	0.530245

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.19518	nan
auc	0.961384	nan
f1	0.947502	0.530245
accuracy	0.915649	0.530245
precision	0.947368	0.530245
recall	0.947635	0.530245
mcc	0.733011	0.530245

Confusion matrix (at threshold=0.530245)

	Predicted as Major	Predicted as Minor
Labeled as Major	683	187
Labeled as Minor	186	3366

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

20.4 seconds

Metric details

	score	threshold
logloss	0.210298	nan
auc	0.955706	nan
f1	0.944367	0.556604
accuracy	0.9109	0.640632
precision	1	0.995963
recall	1	0.0231445
mcc	0.721369	0.640632

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.210298	nan
auc	0.955706	nan
f1	0.944335	0.640632
accuracy	0.9109	0.640632
precision	0.947816	0.640632
recall	0.940878	0.640632
mcc	0.721369	0.640632

Confusion matrix (at threshold=0.640632)

	Predicted as Major	Predicted as Minor
Labeled as Major	686	184
Labeled as Minor	210	3342